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^ ■ ■ • INTRODUCTION 

The terminal goal o£ this project is to produce an 
•individualized multi-media course in Leadership, . Psychology 
and Management, and to document the model used in the course ' 
• development; The attainment of this goal, involves the empirical 
validation of instructional effectiveness through successive 
revision cycles, and the establishment of .an effective system 
to implement the course. 'In addition,, a series of research 
questions Were investigated that had a beari-ng on the materials 
and .the system itself. These varied efforts are often in 
conflict .with each other. That is, in order to obtain reliable 
■data for the validation of instructional effectiveness, re- 
rstrictions mus t .be placed on the operational system. The 
■inclusion of research in an ongoing course adds even further 
restrictions on the freedom of the system to operate'at its 
full potential. .If an individualized multi-media course system 
_ is to demonstrate its full potential for the student, the 
instructor and the educational environment in which it is 
■being used. It eventually must be evalur.ted with all restrictions 
eliminated. 

The full potential of the individualized multi-media 
-course system can.be evaluated from two basic viewpoints. 
•First of .all,, it should . attempt to optimize performance for 
each student on thebasis of empirically reliable data. 



It should be cautioned, howeveri that the ostablishmcn'.. 
of an elaborate system to optimize instruction for each 
individual that is not based on hard data that is reliable 
and significant or that is unwieldy to implement may produce 
dissatisfaction with the entire course.. In essence the 

•optimization procedures must be educationally sound and realistic 
in terms of the capabilities and desires of the users of the 
system--instructors and students- alike . ; ^ 

Secondly, the system should free the instructor from 

.being a mere disseminator of information a^nd allow him to 
interact in a meaningful manner with individual students when 
they are in need of his assistance for remediation or enrich- 
ment. It should allow for his interaction with small groups 
of students to integrate information and concepts that have 

■been effectively .self- taught with validated "programed'* 
materials.- The instuctor's lecture then becomes a structured 
discussion session that motivates and brings about the inte- 
gration and transfer of important concepts and principles. 
With .the assistarxe of a data clerk for administrative. purposes, 

-an instructor in this system should be able to manage well over 
100 students and still provide individual assistance "that 

would exceed that of a conventional class of thirty students. 

The initial implementations of the course in the spring 
and fall semesters of 1970 have generally validated subsystems 

and the basic plan of operation for the Leadership Psychology 



and Management course. In essence, the final course system 
• .to be installed in the spring o£ 1971 reflects successive 
refinements made on the basis of the two previous administra 
tions of the course. 



I COURSE SYSTEM DESIGN 

CONTENT 

■ . Core Instrm:tionaVKIatcHals - The refinements to the 
.course content and structure proceed, along two basic lines: 
1) The refinement of core effectiveness and, 2) the develop- 
ment of depth >core and enrichment. The first point of emphasis 
involved the revision of existing, core materials to ensure a 
criterion- level of mastery of the content by the midshipmen. 
Both performance and attitudinal data from the students / as well 
as input from' subject matter experts were utilized in achieving th 
goal". The basic core content consists of twelve parts ^..dch 
-build logically upon each dther and which reflect major content 
areas. 

Leadership, as -a teria, has different meanings for 
different people, and is used by many interchangeably with 
the term management. Part I of the course briefly presents 
theoretical bases of leadership, 'and evolves therefrom a view 
of leadership as a dynamic interaction process involving a 
leader, his followers and an environmental situation. Part I 
further delineates the use of the basic principles of 
psychology and management as components of naval leadership. 
Acceptance of a traditional deep sense of moral responsibility 
as an essential trait of a naval leader is str'essed here and 
intermittently throughgut the course. 

The naval leader will most often be dealing with groups 
of individuals, his subordinates or followers, so he should be 



aware' of the effect of an individual's membership in a group. 
A fundamental knowledge of individual psychology and group 
dynamics is provided in Parts II and III. 

No leader, regardless of his intelligence, or the 
originality or brilliance of his ideas, can be successful 
until he can communicate those- ideas and directions effectively. 
Part IV is designed to impart to the midshipman a knowledge 
of the principles of communication so that he will be effective 
in his; role as, a leader. 

A naval lead<-r will, in his daily routine, be working 
with men, mone^', materia?, and time. The effective use of 
resources available to him can best be achieved by the leader 
who can apply the Management Process. Part V provides the 
.midshipman with the fundamentals of the Management P.rocess. . 

The latter parts of the course concentrate more 
specifically on those aspects of leadership which are peculiar 
to the military. 

The naval leader, as an appointed leader, has an 
official authority foundation in his commission or appointment. 
Moral responsibility for mission accomplishment, subordinate's 
welfare and general efficiency is* perhaps more intimately " 
related to the military leader than to leaders in other areas 
of endeavor. The general theories of authority , • the responsi'- . 
bility coincident with authority, and proper delegation of 
authority are treated in Part VI. 
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A leader's behavior and style are variables in any 
leadership situation. The spectrum of leader behavior, the 
variety of styles, and the causes/effects of the variations 
are explained in Part VII. From such knowledge, the 
midshipman can formulate a personal concept of possible be- 
havior and style for himself. 

the effectiveness of a leader and the influence he can 
exert on his followers depend to a large extent on the relation- " 
ship he establishes with" the subordinates. Building on the 
foundation acquired in Parts II and III', 'vkrt VIII delineates 
specific guidelines for the leader in his relationship with 
his subordinates. The counselor role of a leader, among his 
many responsibilities for the welfare of .his men, is emphasized. 

Part IX treats the subject of morale and esprit de 
corps, their importance to an effective organization, and 
how a leader's actions influence the individual followers 
arid tho group collectively. 

The ability to instill a willingness in one's self 
or one's followers to act correctly under adverse 
conditions is essential for a naval leader. The most 
desirable situation exists in a military organization 
when each member willingly chooses to do the right thing rather than 
because of fear of the consequences of not doing it. Such is 
the nature of discipline as treated in Part X - the positive 
aspect of discipline as opposed to the negative or punitive 
connotation. 
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An organization, properly managed by appointed 
leaders, prepares for the continuity of leadership of that 
organization. Essential to the selection and training of 
leaders, as well as the maintenance of high standards is a 
personnel evaluation process. Par-t XI provides" the midshipmen 
with a knowledge of the system by which he is himself, 
evaluated and introduces him to the role of evaluator which 
he must fill as a leader. 

Part XII, Applied Leadership, serves as a review and 
i;ynthesis of the course. Situational examples are provided 
for the midshipman to observe the application of material 
learned or to apply his acquired knowledge by selecting 
appropriate actions or decisions for the leader in the 
example. . - 

Table I is a graphic representation of the relation- 
ships among" the parts of the course. Reading from bottom to 
top will indicate the order in which the parts are taught. 
There exists within each part, a variety of segments, each 
of which consists of approximately 50 minutes of instruction. 
The segment titles, by part, are listed in Table 2. 



TABLE 1 
COURSE STRUCTURE BY PART 



xn 

Applied 
Leadei'ship 



K 




3 






XI 


Morale and 
Esprit de Corps 




Discipline 




Personnel 
Evaluation 


I 


'- 1 


r— 










- 1 


VI 




vn 




vni 


Authoritj' and 
Responsibility 




Leadership 

Behavior and 
Style _ . 




Senior 

Subordinate 

Relationships 


1 





















V 

Military 
Management 



IV 

Achieving 
Effective 

Communication 



n 

Individual 
Behavior 



m 

Group 
Dynamics 



I 

Overview of 
Leadership 



TABLK 2 

♦ ■ COURSE CONTENT 

PART ONE; OvG^vic ^^.• oF Uadgrs{iiiD 

Segment I: Contcpr of Leadership 

Seflment II: Stano'ai ds of Leadership in the Naval Service 

PART TWO: Individual Rc- navior 

Segmerif I: . Introducfion to Psycliology 

Segment il: . Behavior and Its Observation 

Segment III: Learning . - 

Segment IV: Factors AfFectlng Learning 

Segment V: Attention and Perception 

Segment VI: Motivation 
Segment VII: Conflict 

Segment VIll: Neurotic and Py/chotic Pveacfions 

Segment IX: Personality 

PART THREE; Gr6up Dynamics 

Segment I: Characteristics of Groups 

Segment II: The Relation of the Leader to the Group 

Segment III: Group Interactions 

Segment IV: Conformity as a Factor of Group Behavior 

Segment V: Relation of the Individual to the Group 

PART FOUR: Achieving Effective Communicotion 

Segment I: Importance of Interpersonal Communication 

Segment II: Types of Communication 

Segment III: The Communicaflon Process (Receiver and Borriers) 

Segment IV: The Communication Procejs (Sender and Feedback) 

Segment V: For v^ommunication and Its Dimensions 

Segment VI: Informal Communication 

Segment VII: Communication Under Battle Situations 



PART FIVE: Military A^anagement 



Scgrnenl I: 
Segment II: 
Segment III: 
Segment IV: 
Segmc;nt V: 
Segment VI: 
Segment VII: 
Segment VIII: 
Segment IX: 
Segment X: 



Inlroduction to Management end the Management Proce 

Decision Molcing and Creativity 

Ob|ectivos 

Planning 

Organizing: Principles ond Process 

Organ i 2 i ng : Stru c tu re 

Organizing: Charting 

Directing 

Controlling 

Coordinating 



>MRt SIX; Aujhoriry one! Rosponsib ililY • 
Segment!: Concept of Auihorily ^ 
Segment II: Wliyfoopic Acccpt/Reslsf Authority 
Scgrricnt'lll: Delegation of Aufhorlty; Line-Staff Relaiionship 
Segment- JV: Responsibility ' . . 

PART SEVEN; Leadership Behavior and Style 

Sogmont I: Leadership Behavior ' " 

Segment 11: Leadership Style . 

Segment III: Determiners of Leadership Style The Leader 

Segment IV: Detenniners df Leadership Style The Group and The Situation 

Segment V: ' Participative • Leadership 



PART EI GMT: Senior-Subordinaje Relgtibnships 

Segment I: Grgariizational Structure & Sqcial Distance in Senior- 
Subordinate Relationships 
Segment 11: Officerr^Ehlisted Relationships 

Segment III: Assumption of Command and Formal & Informal Leader Relationshi 

Segment- IV: Jritroduction to Counseliftg 

Segment V: The Counseling Process 

Segment VI: Relations v/ith Seniors and Contemporaries 

PART NINE: Morale - Esprit dc Corps 
Segment I: Morale ' _ 

Segment II: Group Solidarity and Esprit 

PART TEN: Discipline 

Segment I: . Introduction to Discipline 

Segmen! II: Development and Maintenance of Discipline 

PART ELEVEN: Personnel Evaluation 



Segment I: The Role of Evaluation 
Segment II: Enlisted Performance Evaluation 
Segment III: Officer Evaluation 



PART TWELVE: Ap plied Lea dership 

Segment I: Measurement of Effeciivc Leadership 

Segment II: Generally Recognized Characteristics of an Effective Leader 

Segment IH: Techniques of Assuming Command 

Segment !V: "That's an Order I" 
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Depth Core and Enrichment - In this course, 
with its orientation to the self-instructional method, 
some serious questions were raised as to the • ' 

absence of guided group discussions which most USNA 
instructors deemed imperative. Secondly, since self- 
instructional material tends toward a linear, or flat organi- 
zation .of materials , some concern was raised as to the 
opportunity for students to integrate on a higher plane the 
principles across many segments. Hence, for this course, 
the concept depth core ^evolved from a "look in depth" at 
sections of core to- a mechanism where group discussions 
would allow for the integration. and transfer of information 
that the students had achieved on their own in individualized 
sessions. 

Two of these depth-core sessions were utilized in the 
first implementation of the course,- and several others were • 
employed in the fall run of the course. It is expected 
that for the spring "71" implementation of the Leadership 
course, sixteen depth-core sessions will be available. 
The titles of these depth core sessions and. their recommended 
points of usage are given in Table 3. 
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TABLE 3 
DEPTH CORE 
NUMBER ASSOCIATED PART TITLE 
1 Part II 



Pei-ception and Leadership 
Brainstorming and Group Pressure 
Leavitt-Mueller Experiment 

4 Part IV Techniques of Communication 

5 ..f Y Preparing a Plan 



2 Part III 

3 ■ Part IV 



6 Part V 



8 Part VIII 

9 Part VIII 



11 Part XI 

12 * ■ ; 



13 * 

14 * 

15 * 

16 ' * 



Team Management Game for Allocation 
of Resources 



Paft- V- Allocation of Resources in a Crisis 

Situation 



Counseling, Role Playing 

Responsibility of Junior Officer in 
Counseling 



10 Part XI ■ Promotion Review Board 

Design of New Enlisted Evaluation Form 

Group Ordering of Critical Management 
Factors 



Race Relations 
Drug Problem 

The Leader and Situation (s) 
The World of Charlie Company 



Depth core topics which can be used at any point in the 
course. / x- . 
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The depth core sessions are optional for the instructor. 
•.That is,, he may use as many as he deems necessary. The 
sixteen depth core are more than can be used in one semester, 
but they do give the instructor a variety o£ packages from 
which to choose. 

- , Enrichment units imply individual study of discrete 
areas of subject matter, usually a project or experiment 
•resulting in a written report or statement of findings. 
Our approach has been different in that the subject matter, 
seemed to lend itself to the pursuit .of case studies through 
a structured discussion. Hence, over the past eighteen months, 
a- series of Structural Communication modules have been 
developed. At present there are eight, each of which uses 
three to four problems. Since each module, in its present 
format, takes two and one-half to three hours for completion 
we recommend that after content sign off by the USNA.-they 
be- modularized into 30 individual units for enrichment 
requirements. 

COURSE CONFIGURATION 

Impac t of Research '^n Init _ia2_Course_Je^elopment - 
•.In addition to tiie development of the. basic core of 59 segments 
of instruction, WLC developed a scries of instructional ' 
research hypotheses which had an impact on the course 
configuration. The research effort has been well documented 
in previous publications CTR-6.3, 6.3a, 6.4, 6.4a, 6.11 and 
6,12a and b) however^ a brief review is provided- below. 
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Research hypotheses were based on concepts presented" 
in A Behavioral Approach to Instructional Design and Media 
Selection, in which a distinction is made between the 
importance of media and presentaiton forms (Tosti and Ball, 
1969). The distinction is that a medium is only the mode 
of transmission of information whereas the truly iirportant 
characteristics of instruction are the forms of presentation 
of the information within the medium. Any medium can be 
described with respect to its capability for varying 
dimensions of presentation. The most basic dimensions of 
presentation which characterize a medium are the form and - 
frequency of the stimulus, response, and management 
decisions. In matrix form these dimensions are: 



DIMENSIONS OF PRESENTATION 





Form 


Frequency 


Stimulus 


Stimulus representation 


Duration 


Response 


Response demand form 


Response demand frequency j 


Management 


Management form 


Management frequency 



mil 



Stimulus representation is the form in which the 
stimulus is presented. It can be written, spoken, or 
pictorial • 

• * « 

Stimulus duration is the temporal stability of the 

, stimulus. Duration can be transient - such as movies and 

I . lectures, .or persistent - such as textbooks or other 

I 

printed matter. 

Response demand form is the type of response the 
subject is required to make. The form of the response 
demanded can be covert, v/ritten, or spoken. 

Response demand frequency is the frequency with 
which the subject is required respond. The frequency 
with which questions are asked within an instructional 



15 



w4 sequence can be high, medium, or low. 

Management form is the type of. remediation the* 
• subject is given. It can be multilevel - remediation by 
differing the levels of response demand frequency within a 
single form; multiform - remediation by a different form 
of presentation; repetition - a simple repeat of the same 
jj presentation; or error diagnostic - remediation by 

branching according to specific incorrect responses. 

Management frequency is the frequency with which 
the presentation is repeated or changed according to the 
need for remediation. The frequency of management can be 
I I ^^igh, medium, or low depending on response demand frequency. 



In line with the assumption that presentation 

variables were the critical elements in instruction 

rather than the medium itself, WLC developed ^ive 

hypotheses, four of which were based directly on 

variations in presentation rather than variations in 

media. The hypotheses were: 

Hypothesis 1: Given both simple and 
complex tasks, transient presentations with 
high response demand frequeacy (HRDF) will be 
ihore effective than transient presentations 
with low response demand frequency (LRDF) . 
There will be no difference in student 
performance between covert or overt response 
demand (RD) presentations, or between 
videotape and audiotape -panelbook media. 

Hypothesis 2: Given both simple and 
complex learning tasks, persistent presenta- 
tion with high RDF will be more effective 
than persistent, presentation with low RDF. 
There will be nc,^ difference in student 
performance betvecn three conditions of RD 
(overt-written, overt^spoken, and covert). 

Hypothesis 3: High RDF will be more 
effective than low RDF with either high, 
medium, or low management frequency. 

Hypothesis 4: If performance is below ^ 
criterion level with low RDF presentation, 
remediation with higli RDF will result in a 
significant improvement in performance. 

' Hypothesis 5: The use of "peer-monitor" 
procedures will significantly improve student 
performance in an instructional sequence. 



Media selected for the first validation of the 
instructional system were based on three major considera- 
tions: research design requirements, multimedia 
requirements, and implementation requirements. 

1. Media included in the instructional research 
.design were selected 'on the basis of their capabilities 

for varying presentation according to requirements of 
the experimental hypotheses. For exajiiple. Hypothesis 1 
required transient stimulus duration dn addition to 
media comparison; therefore, videotapes and audiotapes 
were selected. By using both audiotapes and videotapes 
in parallel modules over the same content, it was possible 
to control all irrelevant presentation variables and 
study the effects of transient presentation across two 
media as well as compare the effectiveness of the two. 
media. 

2. In. addition to selecting media for tfieir 
research capabilities, media were selected which would 
provide diversification in the instructional sequence.* 
Although it was not the intention to directly compare 
media throughout the course, i.e.. Medium A vs. Medium B, 
or Medium C vs. Medium* D, a number of media were included 
so that some statement could be made about student 
performance via those media. . 
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3. The third major consideration in media selection 
was the project plan for implementing and maintaining 
the multimedia i;ystem beyond the design stage. Of primary 
concern were the costs for developing, implementing, revising 
and maintaining the system. For example, material for certain 
media could be programed during the design stage of the 
project but would be prohibitively expensive to implement 
or maintain (revise) after the termination of the contract. 

- Another concern vas that media fit conveniently into 
an individually paced program of studies without creating 
undue logistical problems. In this regard, special attention 
was given to existing facilities'at the USNA, such as 
computer consoles, media centers, etc., and to the project 
availability of those facilities. 

Media included in the first validation in the spring of 
1970 were: videotaped .and audiotaped lec .ures with pahelbooks, 
syndactic text, linear programed text, computer assisted 
instruction, learning activity summaries (bibliographies), 
film and group discussion. 

Table 4 p. esents a summary of the research plan which 
was conducted during the spring 1970 run of the course. In 
implementing this plan a variety of alternate forms for 45 of 
the 59 segments were developed. This resulted in the creation 
of approximately 250. modules of instruction. 
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A module is a particular instructional condition 
used to prepare and deliver materials for a segment, 
identified in terms of the categories of the Tosti-Ball 
(1969) model. Several parallel modules were prepared in 
each segment utilized for research purposes, representing 
variations specified by the experimental designs. The 
different modules of -a segment are distinguishable from 
one another by differences in presentation design and/or 
media, although the content is the same. Specifications' 
of the modules for each segment ar^ outlined in later 
sections of the paper giving the design of each experiment. 

The cumulative posttest (CPT) unit listed in Table 4 
is a group of three or four adjacent segments within a part. 
There were 13 CPT units involving 45 of the 59 segments of 
the course. The primary criteria for grouping segments into 
CPT units were that the segments dealt with similai types of 
content and objectives, and that the instructional sequences 
relating to particular concepts which were initiated in the 
unit would also terminate in the same unit. All segments in 
a CPT unit were developed in the same medium and with the^^" 
same variations in instructional conditions between modules. 

The CPT unit was the fundamental unit of instruction for 
research purposes, providing the framework on which the 
experimental designs were constructed. The students were 
divided into groups assigned to different modules in the CPT 



unit. A student in any one group would thus encounter the 
same experimental conditions in progressing through the 
three segments of the unit, and would take three progress 
checks (PC's), one after" completing his module of each 
segment. After completing the segindnts and PC's all 
students then took the CPT, a test administered to assess 
overall achievement level under the experimental conditions 
represented in the CPT unit. 

In addition to group comparisons of media and prese 
presentation design a series of analyses of the relationship 
of student characteristics to perofrjnance in the course. 

The purposes of research on student variables in 
the present case concerned the prediction of overall course 
performance, and, more importantly, the prediction of 
achievement with particular media and presentation forms. 
Because of the number of conditions of instruction compared 
in the Leadership course, an invaluable opportunity 
was provided for one of the first large scale investigations 
of student-method interactions • To this end, a large battery 
of potentially predictive variables was included in the 
student data base. 

Use of research findings in course configuration modificati 
The results from the first implementation indicated the need 
for some refinements in the research design as well as some 
additional investigations. Cons'e^uently some changes v;ere 
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made to the research plan and it was replicated in the fall 
1970 run of the course. The general nature of these changes 
are listed below. 

In the first implementation of the course, confirmation 
or knowledge of results was not used. It was felt that more 
reliable data needed for revision of materials could be 
obtained without confirmation. For the fall semester 
replication, confirmation within all instructional sequences 
was added. 

In Experiment I the use of videotapes was dropped, 
and the issue of confirmation was experimentally investigated. 

In Experiment II the overt-spoken condition of response 
demand was replaced with a condition where the students 
underline the response they select. 

Relatively high levels of performance v;ere attained 
on the CAI materials in Experiment III. In order to 
determine whether this effect was due to the medium 
or the content programed for it, a parallel "paper" version 
was prepared for both the CAI segments and the aiidiotape- 
intrinsically programed booklet segments. 

With respect to Experiment IV, it was found that 
many students did not need to proceed through the remedial 
sequences. The syndactic text summaries were very effective. 
In order to obtain a more complete test of the effectiveness 
of each of the remediation forms, students were forced 
through the sequences. 



Additional activities ' included a comparison of the 
effectiveness of the use of audiotape scripts in place of 
the tapes, and an assessment of the use of content maps 
in selected segments as advance organizers. The resultant • 
presentation design which was carried out in the fall 1970 
administration of the course is given in Appendix A. 
■ ' Final Course Configuration - In arriving. at the final 
course C9nf iguration, a variety of jnputs were considered. 
The empirical data considered were the research and effective- 
ness data, and student ratings and time. Two equally 
important inputs were administrative ease of use and costs. 
In lieu- of conclusive findings from the empirical data, 
the later two inputs were given considerable v/eight. 

An initial concern was with the overall effectiveness 
of the various media. The research data, the effectiveness 
data, student ratings and student time all supported the 
general conclusion that differences among media, if they 
exist, are no more substantial nor important than variation 
among units in the same media, resulting from variation in 
content programing skill, or test items. There was no' support 
for the contention that segments already programed in a 
particular medium should be recast in another medium. It 
was indicated that with careful programing and revision, 
an acceptable level of effectiveness can be reached in almost 
any medium. Table 5 indicates the variety of media utilized 
and the number of segments programed in each medium for the 
spring 1971 installation run. 
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TABLE S 
FliN'AL COURSE MEDIA USAGE 

Medium 



No. of Segments 



Audiotape/Panei Book ' • 19 *• 

Audiotape/Intrinsically Programed Booklet (IPB) 6 * 

Computer-Assisted Instruction (CAI) 4 * 

Linear Programed Text 9 

Syndactip Text 20 

Discussion Booklet 2 

* These media are "hardware bound." In order "to increase 
ease of use and reduce administrative costs, "paper" 
• versions of these media were prepared for all segments 
involved. - ^ 



^ 



The use of the audiotape media and computer-assisted 
• instruction^being hardware bound media^ raised some questions 
about administrative ease, of use and costs when large numbers 
of students are .involved. In response to this realistic 
consideration by the USNA, the 19 segments of Audiotape/ 
Panel Book were also made available in syndactic text format. 
The six segments of Audiotape/IPB were also made-available in 
a paper version 'by utilizing the tape scripts with the Booklet, 
rinally the four segments of CAI had an alternate paper 
version in the format of intrinsically programed booklets. 

With the basic media issue settled^ the next question * 
to be answered was what to do about the multitude of modules 
across the course. As can be noted in Appendix A there were 



as many as six modules or variations in the presentation 
form for a single segment. The average was four modules 
to a- segment all delivering the same content. It would 
take some rather conclusive data to justify maintaining 
all. modules as the administrative problems and costs 
would be prohibitive. It- was decided that unless the 
data strongly indicated that more than one module for 
any one segment would be equally effective, the single 
most effective module (based on considerations of the 
research and effectiveness data as well as interest-4fld 
time) would be selected. In reducing the large number of 
modules to manageable proportions four basic variables had 
to be considered: 1) the form of the response demand, 
2) the response demand frequency, 3) management frequency 
and 4) the form of remediation internal to the syndactic 
texts. 

Although major differences were not found with 
respect to the' form of response required of the student, 
the trend was that the overt- selected response condition 
was slightly better than either overt- spoken or covert. 
This finding is generally in concert with the major body 
of prior research that indicates that the form of response 
utilized within the, instructional materials should be 
similar to that required of the student on the tests of 
achievement on those materials. In addition, if data is 
needed for revision then the overt response must be demanded 



■ It was found that the high response demand frequency 
condition was generally superior to the low.rcsponse demand 
• frequency condition. A facilitative effect does appear to 
be produced by the insertion of a significant number of 
questions in the instructional materials. 

A less conclusive finding involved the issue of 
management frequency. However, considering all the inputs, 
• ft appears that the most efficient condition was the use 
^of moderate levels of management frequency in conjunction 
^^-l^.^X^Sli response demand frequency.' 

. .In terms of the remediaton .form in the syndactic texts, 
-the p^rogramed sequence version utilizes high response demand 
frequency. Therefore, the module selected from those shown 
in Appendix A within each segment, was one that, where 
applicable, required an overt response, had a high response 
demand frequency and a middle Tevel management frequency. 

The result of this effort was 59 segments of maximally 
iB.ffective instructional programed in a variety of media with 
distinct media.; options available in 29 of these segments. 
.The final course configuration for the installation run 
•in. the. spring of 1971 is shown in Appendix B, the Course 
Activities Chart. In addition to displaying the media " 
.options, the chart also indicates the suggested placement of 
depth core,- administrative testing" and the minimum weekly 
pace to follow.. 
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Materials Design Considerations - The physical \ 
appearance of the instructional materials has changed 
considerably from when they were first prepared for the 
research and validation runs. The requirements of 
research control directed that each module be individually 
packaged to allow for both the changes in experimental 
conditions and the careful control of evaluation and research, 
tests. They physical size of a single student's packet, Ss 
it accumulated over the semester, was prohibitive by any 
normal standard, of operation. With the reduction of the 
number of modules it was possible to make the course more 
physically manageable. Multiple segments were bound in a 
isingle volume. Experiments were made in reducing the 
camera-ready copy from 8 1/2 x 11", standard size, to 
5 1/2x8 1/2", and 7 x 10". Aftejr some initial experimentation 
the decision was made to use the 7 x 10" size. 

The developmental materials when used by some students 
and' appraisal by some reviev/ers were characterized as having 
a quality of sameness. .This, it was felt, was a euphemism 
for what was considered bland by standards of graphic ' 
presentation. An extensive analysis resulted in a redesign 
to include design elements which would both separate frames, 
highlight important summaries, and which would generally 
contribute to the overall appearance of the page and the 
entire book. 
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In a further effort to improve the graphic appeal, 
redesign of the cover page was commissioned which resulted 
in a more contemporary cover design. Each part has a 
different symbolic connotation for the substance of 
material under study. The designs are simple, yet 
attractive and meaningful. 

In an effort to respond to the comment that the 
student as well as reviewers were not completely aware of 
the macrocosmic objectives of each particular part and 
segment, forewords were prepared for each segment* which 
■briefly discussed what was to be. learned in the segment, its 
importance and its relationship to other topics. In addition, 
the literature was culled for appropriate quotations to 
introduce the segments. 

A concern had been voiced about the ability of the 
consultants or reviewers to get a larger picture or structure 
of the course materials; In responding to the latter, content 
maps by objectives were prepared which presented graphically 
the hierarchical arrangement of content within each segment. 
It was realized that since programed material by its very 
format, tends to give a flat organization to content, the ' 
co-ntent maps would be well used as overviews or advanced 
organizers for the student. Viewed as advance organizers 
the maps provide ideational scaffolding for topics of 
subject matter to be studied. Hence, in the final product 
they are placed immediately following the foreword of each 
segment. 



The resulting product is materially attractive, 
substantively accurate and validated, graphically appealing 
and easy to use. 



COURSE SYSTEM MANAGEMENT 

GENERAL CONSIDERATIONS 

Just as the course content and structure has undergone 
successive refinements based on the previous administrations 
of the. course so too his the manner in which the student 
interacts with the instructional- materials . The refinements 
have been made on the basis of the requirements for revision 
and research. Basically, each. student studied a segment of 
instruction and then was tested on his mas.tery of the 
objectives for that segment. If he achieved 80 percent or 
better on this criterion referenced progress check, he proceeded to 
the next segment. If he did not achieve this level, he was 
given specific remediation .instructions and was required to 
retake the progress check. If he still failed to achieve 
the appropriate level of mastery^ he received special tutoring 
from the instructor. To obtain maximally reliable data for 
revision from the first implementation, the midshipmen were 
required to take the progress checks individually in class 
under the guidance of the course administrator. In essence, 
this required the midshipmen to come in after each segment. 
To allow the midshipmen more freedom in the fall run^ they 
were given the progress checks with the materials but they 
still had to come in and have the course adminis tiator grade 
them to determine if they needed remediation. , 
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The research imbedded in the course imposed further 
restrictions on the students* interaction with the' 
instructional materials. In order to accurately assess the 
research questions being asked, separate norm-referenced 
tests (Cumulative Posttests) were developed which the 
students had to take in class at specific points in the 
instruction. The students also had to delay 'remediation 
on segments within a Cumulative Posttest Unit until they 
completed the research test. However, with the completion 
of the research in the fall run, the Cumulative Posttests and 
the resultant restrictions were removed from the course. 

The installation run of the course in the spring of 
•71 will be characterized by maximum freedom for each 
student to proceed at his own pace and to manage his own 
time and place of study. The freedom for the student to 
manage his own instruction is an important motivational 
effect of an individualized instructional system. The 
rather rigid controls imposed- on the students in the 
previous implementations of the course which were necessary 
in order to obtain valid revision and research data will 
no longer necessary. With the removal of the research 
Cumulative Posttests, the students will be required to come 
to scheduled classes only for depth core sessions and 
administrative tests. The progress checks which were 
controlled on site will be used as self tests for the 
students. 



- Thus, the students will grade their own progress 
. checks and determine their requirements for remediation 
and/or tutoring with the instructor. The instructor will, 
of course, be available for any non-required tutoring and 
for assignment of enrichment sessions. Rather than being 
required to prepare thrice^weekly lectures, the instructor 
can concentrate on the integration and transfer of 
important concepts, and he can work with individual students 
as they need his assistance. In addition, the instructor 
will be aided by the course administrator in record keeping 
and scheduling. 

The management of the course system involves three major 
componentr. that interact with the instructional materials and 
media. These three components are the course administrator, 
the instructor, and the students. A discussion of the role 
of each of these groups and their interactions with each 
other and the instructional materials will clarify the 
intended management of the system. 

ROLE OF THE COURSE ADMINISTRATOR (CA) 

The CA provides the interface between the student and 
the materials and between the student and the instructor. 
His raison d'etre is to relieve the instructor of as much 
of the administrative work as possible so that the instructor 
is free to handle more students with greater efficiency 
and effectiveness. 



Responsibilities of the CA include: 

* storage of all cere course materials, depth core, 
enrichment materials., A/V equipment, and test items 

* distribution of all course materials 

* assisting the instructor in scheduling students for 
depth core, administrative tests and CAT and notify- 
ing students of times of required attendance 

* coll.^ction of all forms (progress check answer sheets 
and answer sheets internal *to the ifraterials) 

* recording and reporting to the instructor the 
student's progress through the course materials, 
his performance on progress checks (graded by 

the student), and scoring and recording administrative 
tests 

* keeping both the instructor and student informed about 
the student's progress and grades. 

Materials Distribution - IVhen the student has completed the 
administrative pretest which will be given at the first class 
meeting, the CA will issue him Volume I. The segments included 
in each volume as well as the media available for that content 
are in<'icated in Table 6 . When the student has completed 
Volume i he will return his progress check answer sheets to 
the CA in Room 220, Luce Hall. The CA will record each- 
student's performance data and verify that all forms are 
filled out correctly. If the student failed to achieve at 
least 80% the second time through any progress check, the CA 



TABLE 0 

SEGMENTS BY-VOLUME AND MEDIA EMPLOYED 
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Volume 
Number 



I 

ITT 



Segments 
Included 



Media 



1.1 
1.2 



TT 
2.2 
2.3 
2.4 
2.5 



"ZTT 
2.7 
2.8 
2.9 



3.1 
3.2 
3.3 
3.4 
5.5 

4.' 2 
4.3 
4.4 



4.6 
4.7 



5.2 
5.3 
5.4 



TTT" 
5.6 
5.V 
5.8 
5.9 
5.10 



Syndactic Text 
Discussion booklet 



iyndactic text 
Audiotape/Panel book 
Audiotape/Panel book 
Audiotape/Panel book 
Audiotap e/Pahel hnnV 
ayndactic text 
Syndactic text 
Syndactic text 
S yndactic text 



T^udiotape/Panel booF 
Audiotape/Panel bjok 
Audiotape/Panel book 
Audiotapte/pancl book 
Syndactic te xt 

mear text 
Linear text 
Linear text 
Audiotape/Intrinsi- 
call- Programed 
booklet 

udiotape/lntrinsi- 
cally Programed 
booklet 
Audiotape/Intrinsi- 
cally Programed 
booklet 

Aadiotape/Intrinsi- 
i cally Programed 

booklet 

Syndactic text ^ 
Syndactic text 
Syndactic text 
Linear text 
Linear text ' 
Linear text 
Audiotape/Panel book 
Audiotape/Panel book 
Audiotape/Panel -book 
Audiotape/Panel b ook 



Alternate 
Media 



Syndactic text 
Syndactic text 



Volume 
Number 



Syndactic text 
Syndactic text 



syndactic text 
_Syndacti c Jtext 
Syntactic text 
Syndactic text 



II-B 




IV-A 



Tape Script § 
Intrinsically 
Programed bo oklet 
lape Script § 
Intrinsically 
Prgoramed booklet 
Tape Script § 
Intrinsically 
Programed booklet 
Tape Script a 
Intrinsically 
Programed book let 



Syndactic text 
Syndactic text 
Syndactic text 
Syndactic text 



V-C 



V-D 



TABLE 6 (Continued) 



Volume 
Number 


j Segments 
j Included 


Media 


Alternate 
Media 


Volume 
Number 


VI -A 


6.1 
1 6.2 


Linear text 


— 


VII-B 


VI-B 


6.3 
i 6.4 


Linear text 

oy iiudv-tlC text 




VII-A 


7.1 
• 7.2 
7.3 
7.4 
1 7.5 


Audiotape/Panel book 
Audiotape/Panel book 
Audiotape/Panel book 
Audiotape/Panel book 
Audiotape/Panel book 


Syndactic text 
Syndactic text 


Syndactic text 
Syndactic text 
Syndactic text 


VII-C 


VIII-A 


8 1 
8.2 
1 8.3 


iiinear cext 
Linear text 
Linear text4 






VIII-B 


8 d 

8.S 
8.6 


oynaactic text 
Syndactic text 
Syndactic text 






IX-A 


9.1 
9.2 


Audiotape/panel book 
Audiotape/Panel book 


Syndactic text 
Syndactic text 


IX- 3 


\ 


10.1 
10.2 


Audiotape/ Intrinsi- 

Callv P'roo'TiiinAr? 

booklet 
Audiotape/Intrinsi- 
cally Programed 
booklet 


rape Script 5 
Intrinsically 
Programed boo 

Tape Script § 
Intrinsically 
Programed bokl 


X 

let 
et 


XI 


11.1 
11.2 
11.3 


Syndactic text 
Syndactic text 
Syndactic text 








12.1 
12.2 
12-3 
12.4 


computer- Assisted 

Instruction 

Computer-Assisted 

Instruction 
Computer- Assisted 

instruction 


Intrinsically IXII-A 
Programed booklet 


Intrinsically IXII-B 
Pro.^ramed booklet 


Intrinsically iXII-C 
Programed booklet 


Computer-Assisted 
Instruction 


Intrinsically J 
Programed booklet 
f 



will ensure that the student receives the required tutoring. 
If his own instructor is not available at the moment, the 
student may be tutored by any available instructor, or he may 
arrange a later time for tutoring. The student will be given 
the materials for the next part unless the instructor 
specifically indicates that he v/ants the student tutored 
before he receives the next set of materials. 

For, the convenience of both the student and CA, materials 
will be distributed in sets larger than single volumes. The 
student would have to make 19 trips to th3 administrative 
office to turn in one volume and pick up the next if the 
materials were distributed on a v ^lume-by-volume basis. 
Distributing materials by parts (there are at most three 
volumes per part) \irould seem to be the most logical v;ay of 
handling materials distribution. 

In handing out materials by part, the CA will ascertain 
ahead of time \tfhether any media selection is involved for 
the volumes contained in that part. If there is none, the 
CA will simply give the. student his next set of materials. 
If media selection is nentai led , ho\v'ever, the CA will tell 
the student to check first v/ith his Instrv.ctor about which 
medium to choose. If the instructor has said that the student 
can select the medium on his ov/n, the CA \^ill ask the student 
which medium he prefers. 



Record-Keepinor . The CA will update records of the student 
progress through the. materials, and record his performance 
Records will also be kept by the CA on 

* performance on administrative tests 

* attendance at depth core discussions 

* performance on enrichment materials 

* materials distributed to student 

* checkout and return of audio tapes and players. 
Scheduling - The CA will assist the instructors in deter- 
mining dates and times for testing and depth core, and 
will schedule the students for these activities. Notifi- 
cation of required student attendance (by section) will 

be posted on a bulletin board outside the administrative 
office (Room 220 Luce Hall). 

ROLE OF THE INSTRUCTOR 

One benefit to be derived from an individualized' 
course such as the Leadership, Psychology and Management 
course is that the core content is delivered via 
instructional materials which have undergone extensive 
revision and validation and have been proven effective. 
The instructor is thus freed from the normal routine df 
determining what content to teach and how best to teach it, 
and no longer has the thrice-weekly task of preparing and 
delivering instructional lectures. 



The instructor will generally meet with a 'student 
after he has completed a certain area of work. He can 
then clarify for him any points which gave him trouble, 
discuss the relation of the content area to leadership 
problems which the student may currently have or may 
expect to face in the future, and explore other content 
areas of interest to the student but covered only 
peripherally by the core materials. 

, The instructor has the option of dealing frequently 
with the student on this one-to-one basis to the degree 
that each finds the discussion helpful and has the time ■ 
and interest to pursue them. Alternatively, if a student 
seems to be learning well from the materials (as evidenced 
by his performance on administrative tests and progress 
checks), the instructor need not deal on a close personal 
basis with him. The instructor can regulate the degree 
of interaction with his students, ranging from seldom 
seeing alone the student who is performing adequately 
to frequently seeing the student who has difficulty with 
or great interest in the content of the course. 
Instructor-student consultations can be initiated by 
either participant or by some predetermined criterion 
(e.g., the instructor insisting on seeing any student who 
gets a "C" or below on a test.) 

It should be apparent that by releasing the instructor 
from the taslc of developing and teaching core content, 
more time is available for him to engage in other, more 



meaningful, educational contacts with his students on an 

individual basis. Established points of interaction with ' 

the students are: a) the first day of class, b) 

assignment of media, c) system required tutoring, 

d) depth core and enriclment sessions, and e) administrative 

testing. 

The First Day of Class .- The first class meeting should be 
devoted to a complete explication of the course, the 
mechanics of procedures, and the expectation that is 
placed upon the student in this self-paced individualized 
instructional course. Since most courses that the 
student has been exposed to are lock step and require 
little self-discipline, great care should be taken that 
the students understand their responsibility to this 
innovative approach to instruction. The orientation 
session is perhaps the most important meeting of the 
entire course. At that time, student guides will be 
handed out and discussed. It is recommended that the students 
be told to contact the CA to answer any questions 
relating to the student guide and to clarify up any 
administrative questions on procedures. As documentation 
for the instructor's role in these opening sessions, an 
instructor guide will be available with suggested 
procedures for opening day activities. Instructions 
should be given to the students to see the course adminis- 
trator in Room 220, Luce Hall to take the administrative 
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pretest before taking out the first volume. The 
students should be informed that the pretest is intended 
to assess their entering level of knowledge and will not 
be used in grading. After completing the pretest, they 
will be if.sued Volume I, and they may begin proceeding 
at their own pace. 

Assignment of Media - When the instruction is offered 
in more than one medium, the instructor may want the 
student to cpnsult with him about which medium over 
particular content areas may be more profitable for 
that student. Table 6 indicates the alternate media 
available. The instructor would base his advice on 
the student's previous performance in that medium and 
on the student's attitude toward the medium. 

. It is hoped that the cross validation results on 
the relationship of student characteristics to 
performance with the various conditions of instruction 
will be available as additional input in the media 
selection process by the time the student reaches the 
later selection points. 

It is recommended that for administrative ease the 
same medium be assigned for contiguous segments as 
indicated below. For instance, the four segments in 
2.2 through 2.5 are available as audiotape/panelbook or 
syndactic text segments. The student who opts for tapes 
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for these segments will be provided with tapes and 

a tape player by the course administrator who will see 

that these items are checked out and returned by the 

student. Permitting the student to select tapes in 

2.2 and 2.4 and syndactic texts in' 2.3 and 2.5 with 

corresponding bookkeeping, would increase the CA's 

job considerably and would risk tying up tape players 

(which are in limited supply) if they were not returned 

promptly. Furthermore, if the instructor wishes to 

advise the student on media selection, only six student- 

■ instructor consultations would be necessary if media were 

assigned over contiguous segments. These consultations 

would be needed at the following points: 

When student has Discuss media selection 
completed Volume ; for segments: 

I .2.2-2.5 

II- D 3.1 - 3.4 

III- C • 4.4 - 4.7 
V (A or B) 5.7 - 5.10 
VI-B 7.1 - 7.5 

VIII-B . 9.1 - 9.2 

10.1 - 10.2 
*12.1 - 12.4 

* Early selection needed to permit advance CAI scheduling. 



If the instructor does not wish to participate on 
these six occasions in the student's media selection, he 
may have one meeting with each student at the beginning 
of the semester to determine the student's media selections 
for the entire course. This has the drawback that the 
student may find himself committed to use a medium which 
he later discovers he does not like. 

Alternatively, the instructor may want to leave media 
selection for some or all of the blocks of segments entirely 
up to the student. 

WLC will provide forty copies of each tape used in the 
course. To prevent the occurence of a situation in which 
the instructors assign more than forty students to 
the same tapes at the same time, the CA will allot an 
equal number of copies of each tape to each instructor. The 
instructor may then allocate them among his o\m students. 
If an instructor is not going to use his full allotment, 
and another instructor could use "m'ore^ they can easily be 
reassigned by the CA. If an instructor permits his students 
to decide for themselves the media they want to use, the CA 
will distribute the tapes on a first-come-first- served basis. 
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Tutorin£ One of the assets of the system as now conceived 
is that a student should be able to receive tutoring or 
assistance from an instructor at any time during normal ' 
office hours. The student will be more prone to drop by 
and talk over problems with the instructor if he feels 
unfettered by the nuisance of having to arrange to see 
him at a certain time.' Since >M1 instructors' should be 
equally familiar with the content, any instructor would 

tutoring sessions with another 
instructor's students. ~ :• - 



-. -,If a- student fails to achieve a score of 801 or 
better. after taking the progress check twice, he. is 
required by the instructional system to meet with the 
instructor for tutoring. The instructor should go over 
the progress check questions which the student missedj 
dis(?uss vhy one answer is correct, determine why. the 
s.tudent chose an incorrect response, and insure that 
the .student has a correct understanding of the content 
area which caused him difficulty. ...... 



Past experience shows that tutoring time varied from 
ten to thirty minutes. The instructor is requested to 
record (on a WLC-provided log) the time he spends 
per student, per segment in tutoring. * 
Depth Core and Enrichment - The depth core sessions give 
the instructor the opportunity to conduct effective 
guided discussion to integrate and bring about the 
transfer of important concepts. The course activities 
chart (see Appendix b ) indicates .the temporal placement 
of each depth core session, approximating the content 
which the "on schedule" student should have completed in 
relation to the depth core. 

Most of the depth core sessions are introduced by a 
short (five to ten minute) audio or film presentation and 
may have accompanying written materials; these can be 
obtained from the CA. The instructor will determine 
which depth core units he wants to use and will request 
the CA to notify his students of the dates of. required 
attendance. To stimulate discussion and small-group 
intimacy, it is recommended that the instructor divide 
his section of 20 to 25 students in half and require that 
half of the class attend, for example, the Monday class 
meeting and the other half the Wednesday class meeting. 
The CA will post on a bulletin board outside Room 220, 
Luce Hall all pertinent information about required class 
attendance. 



• The instructor may use all or none of the depth 
core units, and may require student attendance or make 
it optional (ho could increase the student's grade for 
each optional depth core discussion that the student, 
attended). Depth core and enrichment packets will be 
outlined in detail and available to the instructor as they 
arc needed. 

• •Administrative Tests - As in any other course, the 
instructor is responsible for the student's grade. 
The-Course Activities Chart (Appendix B) suggests 
possible dates and content areas on which tests can be 
given. These ^osts are spaced' at an average of every two 
weeks although there is no requirement that the instructor 
test at such frequent intervals. 

In constructing a test, the instructor has access to 
several sources of test items: 

* a pool of 1,900 test ite|ns covering the entire 
course; this represents about 25 to 30 criterion- 
referenced test items per segment. 

* almost 500 CpT test items which are more highly 
discriminatitig. All. but eleven of the 59 segments 
were included in the CPT tests; for those segments 

• tested by a CPT test, ten norm-referenced test items 
per segment were developed. 
■ ■ ■ * a file of all the administrative tests used in the 
two validation runs of the course. 
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The instructor is not obligated to use any of the afore- 
mentioned test items (most of which are four-response 
. multiple choice) but may test his students in any way 
he sees fit. It is recommended, however, that the 
instructors be as consistent as possible from section to 
section. 

WLC will provide a standard test answer form foV 
multiple. choice, questions and the CA will assume responsi- 
bility for scoring all non-essay test items and notifying 
the students' and instructor of the grades. 

WLC will also provide an 80- item pretest which i«. to 
be administered before a student is permitted to study 
any of the courso materials. The same 80-items will be 
re-administered at the end of the course either as a 
posttest or as part of the final examination. 
ROLE OF THE STUDENT 

In this individualized system a great deal of 
responsibility will be placed on the student. Although 
he may proceed at his own pace, there will be a minimum 
pace suggested." The course activities chart (Appendix B) will be 
provided for the student (as well as the instructor) 
which will assist him in managing his time. The student 
will be responsible for grading his own progress check 
and determining whether he is in need of remediation 



or tutoring. He will pick up and return the instructional 
materials and related forms from the course administrator 
(CA). He is also responsible for checking with the CA 
for scheduled depth-core and administrative test sessions. 
With the system established as it is for the spring, the 
student should find the instructor more readily available 
on an individual and small group basis in which he can 
gain maximum benefit from the interaction. 



EVALUATION PLANS FOR T»:i: iNSTALLATION PIUSI: 

The initial implementations of the course in the spring 
and fall semesters of 1970 involved the validation of instiac- 
tional effectiveness through successive empirical revision ' 
cycles. During both runs of the course, several research 
studies were conducted. These studies vcre not intended for 
research sake alone. It was hoped that the results would pro- 
.vide insight for revision and further development of the 
course, and to a large degree they have. 

The evaluation plans for the installation phase (spring 
1971) will be aimed at verifying the effectiveness of the val- 
idated materials. In essence, this will be a summative eval- 
uation of instructional effectiveness. In addition, careful 
consideration will be given to the aspects of course system 
operation Anc' management. The course system being employed 
in the installation run reflects successive refinements made 
on the basis of information obtained during the initial try- 
outs of the course. It is at this point then that careful 
evaluation of systems operation and mrnagement has full mean- 
ing. The summative evaluation of instructional effectiveness 
will be discussed first, followed by evaluai:ion plans for the 
course system operation and management. 
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INSTRUCTIONAL EFFECTIVENESS 

Total Course Effectiveness based on .Administrative 
Tests - The evaluation of course effectiveness has been based 
on student performance over two types of criterion referenced 
tests: the administrative pre and posttest and the segment 
progress check. Since both tests consist of items which have 
one-to-one correspondence with the behavioral objectives, it 
is' reasonable to estimate the percentage of objectives attain- 
ed by the average percentage of test items answered correctly. 
To the extent then that test items and objectives have been 
directly matched, the percentage of objectives attained and 
the percentage of test items achieved can be viewed inter- 
changeably. Results of total course effectiveness based on 
administrative tests represent estimates of effectiveness 
from a representative sample of objectives. Descriptive sta- 
tistics, gain score ratios and a test of mean difference will 
be given for the pre and posttest measures. The percentage 
of students achieving various criterion levels will be pre- 
sented. In addition to estimates of total performance, des- 
criptive statistics on total learning time will be given. 

Course Effectiveness Based on Segment Progress Check «; - 
The administrative test represents a sample of total course 
performance. One might reasonably expect a variability of 
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performance across segments of instruction. An in-depth assess 
ment of the systems effectiveness can be obtained from seg- 
ment progress check performance tabulated before and after re- 
medial instruction. The mean percentage of objectives achiev- 
ed and the mean learning time by segment will be given. In 
addition, the percentage of total objectives attained by each 
student (as measured by progress checks), and the mean percent 
across students both before and after remediation will be tab- 
ulated. 

Since the course consists of 12 Parts of fairly distinct 
content, similar types of analyses as done by segment will be 
presented by Part. In assembling the materials, most Parts 
were able to be packaged in a single basic volume - exclusive 
of media options. Some Parts were of sufficient size to war- 
rant multiple volumes. In either case however, a student 
questionnaire was include^ at the end of each basic volume. 

• student questionnaire was included because it is felt that 
in any instructional development and implementation, student 
acceptance is an important factor. One might feel that if a 
student achieves the desired criterion level performance in 
an acceptable period of time, then the instruction is effective. 
However, if in the process of learning the student continual- 
ly finds the instruction lacking .in some respect, then his 
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motivation to approach similar content or presentatiQn forms 
in the future will be diminished. This may be due to the 
specific content itself, the difficulty level, or to the medium 
used t,o present the material. The questionnaire was develop- 
ed to assess these factors as well as general comments about 
the system itself. The student questionnaire is given in 
■ Appendix C. 

Me dia Effectiveness within and across Segment Blocks - 
In addition to the variety'of media provided across the 59 
segments of the course, there are eight points within the 
course, involving 29 segments, where media options are available 
across comparable blocks of segments. These points, the seg- 
ments involved and the media available are presented in Table 7. 

Comparative performance and time statistics as well as 
student reaction will be given for each option point. In ad- 
dition, data will be summarized across segment blocks where 
media options are identical. Finally, summary statistics on 
performance, time and student reaction to each of the sever, 
basic media employed across the course will be presented. Six 
iOf these media are indicated in Table 7 . The remaining medium 
is Linear Text. 
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Option 
Number 



1 
2 



TABLE 7 
MEDIA OPTIONS 



4 

5~ 

6 

7 



Segments 



•2.2-2. 5 



Media Options 



Audio tape/Panel Book; Syndactic Text 

Audio tape/Panel Book; Syndactic Text 

Audiotape/Intrinsically Programed Booklet; 
Audiotape Script/Intrinsically Programed 

Booklet 



3.1-3.4 
!• 4.4-4.7 

5.7-5.10 Audiotape/Panel Book; Syndactic Text 

^jlllll ! Audiotape/Panel Book; Syndactic Text 

I Audiotape/Panel Book; Syndactic Text ' 

10.1-10.2 Audiotape/Intrinsically Programed Booklet; 

i Audiotape Script/Intrinsically Programed 
. I _ Booklet 

12.1-12.4 ' Computer- Ass is-.-.ed Instruction (CAI) ; 

CAI Script (Intrinsic Booklet Format) 



T 
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Use of Effectiveness Data - The types of data discussed 
above as well as being descriptive can provide an intended user 
of the system with invaluable information. The administrative 
tests,, segment and media performance data are also sources of 
information that may be used by the course developer for pos- 
sible revision purposes. Certainly it is the case that possible 
problem -areas may be identified from the above types of data. 
However, unless one wishes to make the mistake of proceeding 
on intuition alone, other types of empirical data should be 
collected. These additional bits of data are the responses 
to individual learning frames, and responses tc individual test 
items assessing achievement on specific course objectives. 
Both of these types of data are being collected and will be 
available to the USNA staff for revision purposes if required. 

INSTRUCT I ON.'M SYSTEM VARIABLES 

Media Options - As was mentioned previously, there are 
eight points within the course involving 29 segments where 
media options are available. The decision process for the 
selection of media for a particular student could be based 
on many factors. It could involve student preference, past 
history of performance, predictions based on individual char- 
acteristics, administrative considerations or some combination 
of these. In lieu of reliable empirical data it would make 
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sense to aljow the student to select the medium he prefers. 
It is often the case, however, that a student will not have 
a strong preference because of lack of information on which 
to base his decision. That is, he may not have had prior 
exposure to the media involved. In this case, student per- 
formance on the initial exposure to various media at the be- 
ginning 'of the course can be tabulated and used in the decision 
process at later points in the course. It should be noted 
here^ however, that this performance includes the effects of 
the content covered as well as the media employed. 

If rel^-able predictors of performance with various media 
are available, a large step towards optimizing the instruc- 
tion for each individual can be made. This is certainly not 
intended to lessen the value of the student selecting the 
media provided he has some data with guidelines for inter- 
pretation on which to base his decision Considering the 
many qualifications necessary in applying complex regression 
equations to a new population of ' students even after cross- 
validation, student preference in media selection might well 
be just as effective. Some of the restrictions involved in 
the use. of the student characteristic data should be noted 
here. 

The investigation of the relationship of student 
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characteristics to performance in. the course was in essence 
added on to a complex research design involving group compari- 
sons. If .the sole. objective was the investigation of student 
characteristics, the research design would have looked quite 
different. Secondly, the student characteristics were related 
to performance on the" norm-referenced Cumulative Posttests 
CCPT^s) In. order for this information to be used effective- 
ly during tlie installation phase, 'the precise relationship 
«•£ performance on the CPT's to performance on segment and ad- 
ministrative test must be determined. Finally^ it should be 
ttoted that a direct comparison of th& relationship of student 
characteristics to performan.ce with audio tape/pane I book as 
opposed to syadactic text is not avail-able.. The syndactic 
texts for media options I, 2^ 4:^ 5, and shown in Table 7 
v.cre..not in use during the first two runs- of the course.. In 
addition ao direct compaxisons of tJie media involved in media 
options 5^ 7- and a: were made because of the- small number of 
students utilizing one of tJie- media' in each case.. This does 
HQ't preclude^ however,, tb.e use of tJie data on the- relation- 
ship of student character r.s:ti:cs to performance with audio- 
tajie/'panel book.,, for example.. Although we may be able to 
•predict triat a particular student would perform poorly on audio- 
tape/'panel book ^ we cannot necessarily say tJiat he would do 
he-tter with syndactic text.. This does not preclude the use 
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of the student characteristic data in attempting to optimize 
performance, but it does mean that we should be aware of the 
limitations. In any case, the decision process and the type 
of information involved will bo reported. Due to the limited 
number of tape recorders available the number of students able 
to select the audiotape option at any one point will most 
likely be limited to 30 to SO students. 

Depth core and Enrichment - Since depth core is option- 
al for the instructor and enrichment is optional for the stu- 
dent, these topics are included under instructional system 
variables. Depth core and enrichment sessions will be evalu- 
ated as they are used, ^jihere appropriate, the evaluation 
procedures for these sessions which will be outlined in detail ' 
as they are delivered will include attitudinal data from students 
and instructors as well as performance data. 

The Instructor - The instructional system frees the in- 
structor from being a mere disseminator of information and 
allows him to interact in a meaningful manner with individual 
students when they are in need of his assistance for reme- 
diation or enrichment. It allows for his interaction with 
small groups of students in depth core sessions to integrate 
information and concepts that have been effectively self- 
taught with validated programed materials. The amount of time 



57 



that an instructor engages in tutoring, depth core and en- 
richment session with the students will be tabulated. This 
type of data is essential in attempting to estimate the num- 
ber of students that any one instructor can handle. Finally, 
the instructors will be asked to keep a continual record of 
their assessment of their role in the system, comparing what 
was established with what actually occurred and noting any 
discrepancies or problems. This will allow for documentation 
of problem areas in the system that were not anticipated at 
the beginning of the course. 

The Course Administrator - The major responsibility of 
the course administrator is to relieve the instructor of as 
much of the administrative work as possible so that the in- 
structor is free to handle more students with greater effi- 
ciency and effectiveness. The amount of time that the course 
administrator is involved with distribution and collection of 
materials as opposed to record keeping and reporting of data 
will be tabulated. The course administrator will also be re- 
quired to keep a comment log to document any problems that 
might arise. 

Record, keeping and Data Management - Control of the 
distribution of the introductory questionnaires, the pretest, 
the instructional materials, and tapes and tape recorders is 
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maintained by means of the Materials Distribution Form. A 
sample page of this form is given in Appendix D . The media 
selected at each option point is given in order to ensure 
that the proper volumes have been handed out. 

Each student records his answers to segment progress 
checks on the Progress Check Response Form (See Appendix E) 
and grades it himself to determine if he needs tx) remediate 
and re-take .the progress check. When this form is turned 
in by the student, the course administrator records the per- 
formance on the Progress Check Performance Record. A sample 
of this form is given in Appendix F . The students are 
listed by section so that ah instructor can check on the overall 
class performance with relative ease. In addition, 
the media chosen and the average performance across" segments 
within eadi media option will be tabulated. 

As can be noted on the Progress Check response form, 
the students are asked to indicate the total time spent on 
each segment (see Appendix e ). An individual student's 
learning time is recorded on the Segment Learning Time Record 
(See Appendix g ). As was done with the Progress Check 
Performance Record, the media chosen and the average time 
spent across segments within each ncdia option will be tabu- 
lated. 
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If a student fails to achieve 801 on any progress check, 
he must remediate and re- take the missed items. If he again 
fails to achieve 804, he must see the instructor for tutoring'. 
In this case the instructor will, be giv^n a form indicating 
the questions missed, and after the tutoring session is com- 
pleted, the instructor will indicate the time spent. The 
tutor form will be returned to the course administrator and 
the time will be recorded on the Tutor Time Record (soe 
Appendix H). 

All of the above forms will be available for the in- 
structors' inspection in the administrative office. In ad- 
dition to these forms, a weekly report will be given to the 
instructors indicating his students' pace through the course. 
Those students who are behind schedule will be clearly noted. 
The weekly performance record is shown in Ai^pendix I". 

The use of the above mentioned forms allows for a sim- 
plistic and efficient management system that does not require 
the use of a computer. However, the forms are set up in such 
•a manner that the conversion to a computer managed system 
could be made. The question yet to be investigated is whether 
it would be cost effective. 
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Volume niuiibci ? 



For the followinfi questions circle the appropriate number. 

Above Below 



Was the material interesting? 

Was the material difficult? 

Rate your approval of the way the 
material was presented (clicck only 
the media used in tliis Volume). 

a. Audiotape/Intrins ically Programed 
Booklet (IPB) 

b. Audiotape Script/IPB 

c. Audiotape/Panelbook 

d. Computer Assisted Instruction (CAI) 

e. CAI Script/IPB 

f. Discussion Booklet 

g. Linear Text 

h. Syndactic Text 



Hi Kb Avg Avg Avg 
1 2 3" ~T" 

12 3 4 



2 
2 
2 
2 
2 
2 
2 
2 



3 

3 
3 
3 
3 
3 
3 
3 



4 
4 

4 
4 
4 
4 
4 
4 



hPJ>' 
5 " 



5 
5 

5 
5 
5 
5 
5 
5 



If your answer to number 4 was "Below Avg." or "Low," identify 
the media and indicate why. 



Have you had any difficulties witli regard to: 

a. Maintaining the minimum pace as indicated in the 
Course Activities Chart? 

b. Following procedures as outlined in the Student 
Guide? 

c. Obtaining instructional materials? 

d. Meeting with the instructor? 

If you answered "Yes" to any portion of number 6, please 
specify the area and the problem. 



Yes No 



If you have any additional comments to make with regard to the 
above questions or any other aspect of the course, please do so 
here. 



MATIiI{IALS UISTPJliUTION CONTROL FOlliM 



NAME 



Albert 



• 2.2-2 5 
Gen. Stu. Pre- Media *^ 
ShXTIONQues. Qucs'.Tcsl Scl.'^^Vol.I 



Boost 



Bradley 



B rownsbor gc> 
Carlson 



Clarkin 



4 

T 



Clifford 



Coleman, A. 



Coleman, D. 



401 



201 



701 



201 



GOl 



701 



601 



701 



401 



201 
40f 



Collins 



Cooper 



Covington 
Coyle 



Crook 



Davldsson 
Dentler- 



Dillon 



Dohse 



Donlan 



Drawneck 
Endicott 

Engluncl 

Evans 



Feltes 



Fleming" 
Frazlcr 



Gallv.p 



Gilbert 
Goodwin 



Griffiths 

Hall 

Ilarrison^ 
tlarrofd 



Hartlcv 



}Ieardincr_ 



Henry 



Hill 



Hoffmann 



701 



501 



701 



501 



201 



501 



701 



501 



201 



501 



601 



501 



201 



1/ 



^y 



< • 
r M 



ft I 

O M 
>M 
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2.2-2. S 



601 



701 



501 



401 



201 
50l' 



201 .. 



701 



701 



401 



501 



501 



201 



201 



701 



401 



1^ 

~7~ 



y. 



y 



S 



S 



S 



s 



! i/L 

y 



y 



s 



5. 



-5. 
S 



J5 



y 



y 



^y 



y 



y 



y 



y 



y 



y 



y 



y 



^y 



A. 



JL 



y 
y 



\y 



y 



y 



y 



y 



x/ 



.yL 

y 



y 



\y 



y 



y 



^y 



y 



y_ 
~\y 



y 



y 



y 



Vol. ■ Taj) OS 

J I -A DTTt " nr 



y 



V 



syl 



x^ 



1/ 



1, 



x^ 



_JU;c or X 

__0irr"i'i 



^y.a.. 



L 



_.rii. 



A - Audiotape/panel book 
B - Syndactic text 
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PART 



SEGMENT 



Name 



ID 



Tofal time spent on' segment (including Progress Check and RemeJiaf Jon) 



minutes. 





TEST 


RESPONSES 


bditrrM atti/^JwI nnrrrir'K r*?* 

RtMEDIATlON KF.FERENCE^ 




FIRST 
TRIAL 


SECOND 
TRIAL 


4..1 Linear Text, Volume IV-A 


- 


1 

1 


• 




Paees 3''6 

" w» V «p/ Vr 




2 






Pages 2-6 




3 






Pages 7-10 




4 






Pages 11-14 




5 







Pages 14-16 




6 




; 


Pages 16-19 




7 
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